As the amount of content available over the Internet continues to grow exponentially in size, the task of finding relevant content has become increasingly cumbersome. Further, such content may not always be sufficiently organized or identified, thereby resulting in missed content.
With the abundance of multimedia data made available through various means in general and the Internet and world-wide web (WWW) in particular, there is a need for effective ways of searching for, and management of, such multimedia data. Searching, organizing and management of multimedia data can be challenging at best due to the difficulty involved in representing and comparing the information embedded within the content, and due to the scale of information to be checked.
Moreover, when it is necessary to find content by means of a textual query, some existing solutions revert to various metadata that textually describe the content of the multimedia data. However, such content may be abstract and complex by nature and not adequately defined by the existing and/or attached metadata.
The rapidly increasing multimedia databases, accessible for example through the Internet, calls for the application of new methods of representation of information embedded in the content. Searching for multimedia is challenging due to the large amount of information that has to be priority indexed, classified and clustered. Moreover, existing solutions revert to model-based methods to define and/or describe multimedia data.
However, by its very nature, the structure of such multimedia data may be too abstract and/or complex to be adequately represented by means of metadata. The difficulty arises in cases where the target sought for multimedia data is not adequately defined in words, or by respective metadata of the multimedia data. For example, it may be desirable to locate a car of a particular model in a large database of images or video clips or segments. In some cases, the model of the car would be part of the metadata, but in many cases, it would not. Moreover, the image of the car may be at angles different from the angles of a specific photograph of the car that is available as a reference search item. Similarly, if a piece of music, as in a sequence of notes, is to be found, it is not necessarily the case that in all available content the notes are known in their metadata form, or for that matter, the search pattern may just be a brief audio clip.
Searching multimedia content has been a challenge for a number of years and has therefore received considerable attention. Early systems would take a multimedia data element in the form of, for example, an image, compute various visual features from it and then search one or more indexes to return images with similar features. In addition, values for these features and appropriate weights reflecting their relative importance could be also used. These methods have improved over time to handle various types of multimedia inputs and to handle them in an ever-increasing effectiveness. However, because of the exponential growth of the use of the Internet, the multimedia data available from these existing solutions have become less effective in handling the currently available multimedia data due to the vast amounts already existing as well as the speed at which new data is added.
Searching through multimedia data has therefore become a significant challenge, where even the addition of metadata to assist in the search has limited functionality. First, metadata may be inaccurate or not fully descriptive of the multimedia data, and second, not every piece of multimedia data can be described accurately enough by a sequence of textual metadata. A query model for a search engine has some advantages, such as comparison and ranking of images based on objective visual features, rather than on subjective image annotations. However, the query model has its drawbacks as well. When no metadata is available and only the multimedia data needs to be used, the process requires significant effort. Those skilled in the art will appreciate that there is no known intuitive way of describing multimedia data.
Therefore, a large gap may be found between a user's perception or conceptual understanding of the multimedia data and the way it is actually stored and manipulated by a search engine. The current generation of web applications is effective at aggregating massive amounts of data of different multimedia content, such as pictures, videos, clips, paintings and mash-ups, capable of slicing and dicing it in different ways, as well as searching it and displaying it in an organized fashion, by using, for example, concept networks.
A concept may enable understanding of a multimedia data from its related concept. However, existing solutions are unable to add any real “intelligence” to the mix, i.e. no new knowledge is extracted from the multimedia data aggregated by these systems. Moreover, the systems tend to be non-scalable due to the vast amounts of data they must handle, as many are configured to analyze and reanalyze entire multimedia data elements to identify various part contained therein. This hinders the ability to provide high quality searching for multimedia content.
It would therefore be advantageous to provide a solution that would overcome the challenges noted above.